Image forming system and image forming apparatus to display together with job information about printing jobs

ABSTRACT

To display a real image forming apparatus together with job information about printing jobs accumulated in this image forming apparatus. A mobile terminal acquires an identification number of an image-captured image forming apparatus by image recognition processing of image data. Further, mobile terminal generates a screen layout corresponding to a detection location of image forming apparatus obtained from the image data. Further, mobile terminal transmits, to a server, an inquiry request for address information, including the identification number. Server has address information about each image forming apparatus. When server receives the inquiry request for address information, it identifies image forming apparatus corresponding to the identification number, and transmits an IP address of identified image forming apparatus to mobile terminal. Mobile terminal uses the acquired IP address to acquire job information about printing jobs from image forming apparatus, synthesizing the job information with the image data for display.

INCORPORATION BY REFERENCE

This application is based on and claims the benefit of priority fromJapanese Patent Application No. 2013-179244 filed on Aug. 30, 2013, thecontents of which are hereby incorporated by reference.

BACKGROUND

The present disclosure relates to an image forming system which includesa mobile terminal, an image forming apparatus, and a server, and animage forming apparatus, and particularly to an image forming systemwhich is capable of confirming a job information about an image formingapparatus, and an image forming apparatus.

With an image forming apparatus, such as a printer, an MFP(Multifunctional Peripheral), or the like, when printing jobs arecharged, the printing jobs are normally executed in sequence in theorder of having been loaded. The printing jobs which have been loadedare accumulated in a storage part in the inside of the image formingapparatus until the turn of being executed comes. What amount ofprinting jobs is accumulated in the image forming apparatus can beconfirmed by displaying information about the printing jobs which havebeen accumulated.

Incidentally, there have been proposed various types of techniques fordisplaying the accumulated data. For example, there has been proposed atechnique which arranges the data stored in the storage part on athree-dimensional coordinate space in accordance with the date ofpreparation and the category, and displays it in the display part.

SUMMARY

An image forming system according to one aspect of the presentdisclosure includes a mobile terminal, a plurality of image formingapparatuses, and a server. A server storage part provided in the serverstores pieces of address information about the plurality of imageforming apparatuses. When a photographing part provided in the mobileterminal is used to photograph the image forming apparatus, an inquiryrequest for the address information about the image-captured imageforming apparatus is transmitted from the mobile terminal to the server.In response to the inquiry request for the address information, theserver storage part is referenced for identifying the image-capturedimage forming apparatus. Further, the address information about theidentified image forming apparatus is transmitted from the server to themobile terminal. On the basis of the address information about theidentified image forming apparatus that has been transmitted from theserver, a request for transmitting a job information is transmitted fromthe mobile terminal to the identified image forming apparatus. Inresponse to the request for transmitting a job information, a jobinformation about the printing jobs accumulated in the identified imageforming apparatus is transmitted from the identified image formingapparatus to the mobile terminal. The job information which has beentransmitted from the identified image forming apparatus is synthesizedwith image data which has been prepared by the photographing part to bedisplayed in the display part of the mobile terminal.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a system configuration drawing of an image forming system ofan embodiment 1 of the present disclosure;

FIG. 2 is a block diagram illustrating a configuration of an imageforming apparatus shown in FIG. 1;

FIG. 3 is a block diagram illustrating a configuration of a mobileterminal shown in FIG. 1;

FIG. 4 is a block diagram illustrating a configuration of a server shownin FIG. 1;

FIG. 5 is a flowchart for the image forming system shown in FIG. 1;

FIG. 6 is an image view illustrating how image forming apparatuses areimage-captured by the mobile terminal shown in FIG. 1;

FIG. 7 is an image view of a screen layout which has been generated byan image recognition processing part shown in FIG. 3;

FIG. 8 is an image view illustrating a situation in which the imageforming apparatuses which have been image-captured by the mobileterminal shown in FIG. 6 are displayed together with pieces of jobinformation about the image forming apparatuses;

FIG. 9 is a block diagram illustrating a configuration of a mobileterminal in an embodiment 2;

FIG. 10 is a block diagram illustrating a configuration of a server inthe embodiment 2;

FIG. 11 is a flowchart for an image forming system in the embodiment 2;

FIG. 12 is an image view of a screen layout which has been generated byan image recognition processing part in the embodiment 2;

FIG. 13 is an image view for a room layout information stored by theserver in the embodiment 2; and

FIG. 14 is an image view of a situation in which an address informationtransmission processing part in the embodiment 2 has replied IPaddresses to the mobile terminal.

DETAILED DESCRIPTION

Image forming systems shown in embodiments 1 to 3 described belowimplement the AR (Augmented Reality) technology. Specifically, in theimage forming system, a mobile terminal is used to photograph an imageforming apparatus, and an image-captured video image is displayed in adisplay part of the mobile terminal. In addition, an address informationabout the image-captured image forming apparatus is inquired of aserver. Further, on the basis of the inquired piece of addressinformation, a job information is acquired from the image formingapparatus. Further, the image forming apparatus and the job informationare both displayed on the mobile terminal.

Hereinbelow, with reference to the drawings, the embodiments of thepresent disclosure will be explained in detail. In the followingembodiments, the component exhibiting the same function is provided withthe same symbol.

<Embodiment 1<

As shown in FIG. 1, an image forming system 100 includes first to nthimage forming apparatuses 1-1 to 1-n, a mobile terminal 3, and a server4. The first to nth image forming apparatuses 1-1 to 1-n, the mobileterminal 3, and the server 4 are connected to a network 2. The network 2may be a wired network, a wireless network, or a network in which awired and a wireless are mixed. In the drawing, the image forming system100 includes n (the symbol n denotes a natural number) image formingapparatuses. However, the number of image forming apparatuses may beincluded one or more in the image forming system 100.

The first to nth image forming apparatuses 1-1 to 1-n are constituted bya printer(s), an MFP(s), and the like, with a mark having anidentification number being provided on, for example, the front face.The identification number may be a manufacturing number (a serialnumber) of the respective first to nth image forming apparatuses 1-1 to1-n. Further, the identification number may be a number which is uniqueamong the first to nth image forming apparatuses 1-1 to 1-n.

A configuration common to the first to nth image forming apparatuses 1-1to 1-n will be explained with reference to FIG. 2. Hereinbelow, in thecase where the first to nth image forming apparatuses 1-1 to 1-n are notdistinguished from one another, they will be explained simply as animage forming apparatus 1, respectively.

The image forming apparatus 1 includes an operation part 11, acommunication part 12, a storage part 13, an image processing part 14, aprinting part 15, a job information transmission processing part 16, anda control part 17.

The operation part 11 includes a display part and an input part. Theoperation part 11 is a user interface including various operation keysfor receiving operations by a user on the image forming apparatus 1.

The communication part 12 has a function to transmit/receive variousdata to/from the mobile terminal 3 or the server 4 through the network2.

The storage part 13 is a non-transitory storage medium. The storage part13 stores image data transmitted from a terminal (not shown) via acommunication part 12. Further, the storage part 13 stores image dataread by a document reading part (not shown). These image data in thestorage part 13 are accumulated as printing job(s).

The image processing part 14 performs prescribed image processing on theimage data. The image processing part 14 performs, for example,enlargement/reduction processing, or image improvement processing, suchas density adjustment and gradation adjustment.

The printing part 15 prints the image data on the basis of the printingjobs stored in the storage part 13. The printing part 15 forms a latentimage on a surface of a photosensitive drum on the basis of the imagedata read out from, for example, the storage part 13. Further, theprinting part 15 performs image formation by using toner to provide atoner image on the basis of the latent image. Further, the printing part15 causes the toner image to be transferred from the photosensitive drumto a recording sheet. Further, the printing part 15 causes the tonerimage to be fixed on the recording sheet, which is then discharged.

The job information transmission processing part 16 has a function torespond to a request for transmitting a job information from the mobileterminal 3. On the request for transmitting a job information, the jobinformation transmission processing part 16 transmits a job informationabout the printing jobs stored in the storage part 13 to the mobileterminal 3 through the communication part 12. The job informationincludes existence/non-existence of printing job, and a number ofprinting jobs. Further, in the case of existence of printing job, thejob information includes an expected execution termination time for allthe printing jobs, and the details of each printing job (for example,the name of a document to be printed, the user name, and the like). Inaddition, the expected execution termination time for all the printingjobs is calculated in accordance with a number of printing pages for theprinting jobs, and the printing processing capacity of the image formingapparatus 1 for executing the printing jobs, and the like.

The control part 17 is connected to the communication part 12, thestorage part 13, the image processing part 14, the printing part 15, andthe job information transmission processing part 16, respectively. Thecontrol part 17 is an information processing part, such as amicrocomputer including a non-transitory storage medium. In the storagemedium of the control part 17,

a control program for performing operation control of the image formingapparatus 1 is stored. The control part 17 reads out the control programstored in the storage medium, and expands and executes it, therebyperforming control of the entire apparatus.

In the case where the image forming apparatus 1 is an MFP, in additionto the above-described configuration, there may be provided parts, suchas a document reading part comprised of a scanner, and the like, and afacsimile transmission/reception part for transmitting/receiving afacsimile signal via a public line network comprised of a modem, and thelike.

The mobile terminal 3 is a device equipped with a camera. As shown inFIG. 3, the mobile terminal 3 includes a photographing part 31, an imagerecognition processing part 32, a storage part 33, a communication part34, a display part 35, a request processing part 36, and a control part37.

The photographing part 31 is comprised of a camera, and the like, fortaking a picture in a direction of photographing to capture image data.The captured image data is outputted to the display part 35 and theimage recognition processing part 32. In the display part 35, thecaptured image data is displayed as an image-captured video image.Further, the captured image data is subjected to image recognitionprocessing by the image recognition processing part 32.

The image recognition processing part 32 performs image recognitionprocessing for detecting the image forming apparatus 1 from the imagedata captured by the photographing part 31. In the image recognitionprocessing, the image recognition processing part 32 references an imagerecognition data 331 stored in the storage part 33 to acquire anapparatus information about the image forming apparatus 1. For example,the image recognition processing part 32 extracts a feature amount forthe image data captured by the photographing part 31, and collates thisfeature amount with the image recognition data 331 stored in the storagepart 33, thereby detecting an image of the image forming apparatus 1.Further, for example, the image recognition processing part 32 acquiresan apparatus information about the image forming apparatus 1 whichcorresponds to the detected image. In addition, the image recognitiondata 331 is data which associates the recognition pattern of the imageforming apparatus 1 with the apparatus information (the identificationnumber, the model information, and the like) for the image formingapparatus 1.

An image of the image forming apparatus 1 that is detected from theimage data is, for example, a mark provided on the image formingapparatus 1. In this case, the image recognition processing part 32detects the mark provided on the image forming apparatus 1 to therebyacquire an identification number of the image forming apparatus 1 as anapparatus information about the image forming apparatus 1.

On the basis of a detection location of the image forming apparatus 1 inthe image data, the image recognition processing part 32 generates ascreen layout representing a display location of the image formingapparatus 1 on a display screen in the display part.

The storage part 33 is a non-transitory storage medium. The storage part33 stores the image recognition data 331. In the image recognition data331, the apparatus information (the identification number, the modelinformation, and the like) about the image forming apparatus 1 areassociated with the recognition pattern (mark) of the image formingapparatus 1. The pattern (mark) for the image forming apparatus 1 isused for image recognition processing in the image recognitionprocessing part 32.

The communication part 34 has a function to transmit/receive variousdata to/from the image forming apparatus 1 or the server 4 through thenetwork 2.

The display part 35 has a display function and an input function. Thedisplay part 35 displays an image on the basis of the image data whichhas been acquired from the photographing part 31. Further, the displaypart 35 acquires a job information about the image forming apparatus 1through the communication part 34, and uses an OSD (On Screen Display)function for displaying the job information, synthesizing it with theimage data in association with the disposition location of the imageforming apparatus 1 in the screen layout. The input part of the displaypart 35 may be constituted by an operation key, and the like, which areseparately provided.

The request processing part 36 transmits an inquiry request for anaddress information to the server 4 in order to acquire an IP address ofthe image forming apparatus 1 image-captured by the photographing part31. The inquiry request for an address information includes theidentification number which has been acquired by the image recognitionprocessing part 32. It is assumed that the request processing part 36 ispreviously provided with an IP address of the server 4, thereby beingcapable of communicating with the server 4.

By the inquiry request for an address information, the requestprocessing part 36 acquires an IP address of the image forming apparatus1 from the server 4. On the basis of the IP address, the requestprocessing part 36 performs a request for transmitting a job informationto the image forming apparatus 1. Thereby, the request processing part36 acquires a job information from the image forming apparatus 1.

The control part 37 is connected to the photographing part 31, the imagerecognition processing part 32, the storage part 33, the communicationpart 34, and the display part 35, respectively. The control part 37executes operation control of the entire mobile terminal 3 in accordancewith a prescribed instruction data inputted from the display part 31.The control part 37 is an information processing part, such as amicrocomputer including a non-transitory storage medium. In the storagemedium, there is stored a control program for performing operationcontrol of the mobile terminal 3. The control part 37 reads out thecontrol program stored in the storage medium to expand it. Thereby, thecontrol part 37 performs control of the entire apparatus in accordancewith a prescribed instruction data inputted from the display part 35.

The server 4 includes a communication part 41, a storage part 42, anaddress information transmission processing part 43, and a control part44, as shown in FIG. 4.

The communication part 41 has a function to transmit/receive variousdata to/from the image forming apparatus 1 or the server 4 through thenetwork 2.

The storage part 42 is a non-transitory storage medium, storing anaddress information 422.

The address information 422 is constituted such that the IP addresses ofthe first to nth image forming apparatuses 1-1 to 1-n are associatedwith the apparatus information (the identification number, the modelinformation, and the like).

The address information transmission processing part 43 receives aninquiry request for an address information from the mobile terminal 3through the communication part 41. When the address informationtransmission processing part 43 receives the inquiry request for anaddress information, references the storage part 42 to identify an imageforming apparatus 1 corresponding to the identification number includedin the inquiry request for an address information. The addressinformation transmission processing part 43 returns an IP address of theidentified image forming apparatus 1 to the mobile terminal 3.

The control part 44 is connected to the communication part 41, thestorage part 42, and the address information transmission processingpart 43, respectively, executing operation control of the entire server4. The control part 44 is an information processing part, such as amicrocomputer including a non-transitory storage medium. The storagemedium stores a control program for performing operation control of theserver 4. The control part 44 reads out the control program stored inthe storage medium to expand it for performing control of the entireserver 4.

The flow of processing in the image forming system 100 configured asabove will be explained with reference to FIG. 5. Here, for explanation,it is assumed that the user photographs the first image formingapparatus 1-1 and the second image forming apparatus 1-2 through themobile terminal 3.

The photographing part 31 of the mobile terminal 3 photographs the firstimage forming apparatus 1-1 and the second image forming apparatus 1-2(Step S11). This generates image data including the first image formingapparatus 1-1 and the second image forming apparatus 1-2. Then, in thedisplay part 35, as shown in FIG. 6, an image is displayed on the basisof the image data captured by the photographing part 31. The user canphotograph the first image forming apparatus 1-1 and the second imageforming apparatus 1-2 while viewing the display part 35. At this time,the user takes a picture such that the marks provided on the respectivefirst image forming apparatus 1-1 and second image forming apparatus 1-2appear in the picture.

The image data captured by the photographing part 31 is outputted to theimage recognition processing part 32. The image recognition processingpart 32 references the storage part 33 for performing image recognitionprocessing (Step S12). On the basis of the mark provided on the firstimage forming apparatus 1-1 and the mark provided on the second imageforming apparatus 1-2, the image recognition processing part 32 detectsthe first image forming apparatus 1-1 and the second image formingapparatus 1-2 from inside of the image data.

For the first image forming apparatus 1-1 and the second image formingapparatus 1-2 which have been detected by the image recognitionprocessing, the image recognition processing part 32 references thestorage part 33. Thereby, the image recognition processing part 32acquires identification numbers of the first image forming apparatus 1-1and the second image forming apparatus 1-2 as respective pieces ofapparatus information (Step S13).

Further, from the image data, the image recognition processing part 32generates a screen layout on the basis of the locations where the firstimage forming apparatus 1-1 and the second image forming apparatus 1-2have been detected, respectively (Step S14). The screen layout includesthe locations where the first image forming apparatus 1-1 and the secondimage forming apparatus 1-2 are displayed on the display screen,respectively.

For example, FIG. 7 is an image view of a screen layout showing thelocations where the first image forming apparatus 1-1 and the secondimage forming apparatus 1-2 are displayed on the display screen,respectively. The first image forming apparatus 1-1 and the second imageforming apparatus 1-2 are disposed on the screen layout in associationwith the locations detected from the image data by the image recognitionprocessing part 32. In FIG. 7, in the disposition location on the screenlayout for the respective first image forming apparatus 1-1 and secondimage forming apparatus 1-2, the respective identification numbers“123AAA” and “456BBB” are given for convenience.

Next, about the first image forming apparatus 1-1 and the second imageforming apparatus 1-2 which have been image-captured by thephotographing part 31, the request processing part 36 transmits aninquiry request for a address information to the server 4 through thecommunication part 34 (Step S15). Thereby, to the server 4, therespective identification numbers of the first image forming apparatus1-1 and the second image forming apparatus 1-2 which have beenimage-captured by the photographing part 31 are transmitted, which isincluded in the inquiry request for an address information.

The address information transmission processing part 43 of the server 4receives the inquiry request for an address information from the mobileterminal 3 through the communication part 41. Then, the addressinformation transmission processing part 43 references the storage part42 to identify an image forming apparatus 1 corresponding to theidentification number included in the inquiry request for an addressinformation (Step S16).

Next, the address information transmission processing part 43 referencesthe storage part 42 to acquire an IP address of the respective firstimage forming apparatus 1-1 and second image forming apparatus 1-2 (StepS17). When the IP address has been acquired, the address informationtransmission processing part 43 associates the identification numberwith the IP address of the respective first image forming apparatus 1-1and second image forming apparatus 1-2. The address informationtransmission processing part 43 transmits this to the mobile terminal 3through the communication part 41 (Step S18).

The request processing part 36 of the mobile terminal 3 receives the IPaddress associated with the identification number of the respectivefirst image forming apparatus 1-1 and second image forming apparatus 1-2from the server 4 through the communication part 34. On the basis of thereceived IP address, the request processing part 36 performs a requestfor transmitting a job information to the first image forming apparatus1-1 and the second image forming apparatus 1-2 through the communicationpart 34, respectively (Step S19).

The job information transmission processing part 16 of the first imageforming apparatus 1-1 receives the request for transmitting a jobinformation through the communication part 12. Then, on the basis of theprinting jobs stored in the storage part 13, the job informationtransmission processing part 16 acquires a job information (Step S20).Specifically, the first image forming apparatus 1-1 acquiresexistence/non-existence of printing job, and a number of printing jobsas a job information. Further, in the case of existence of printing job,the first image forming apparatus 1-1 also acquires an expectedexecution termination time for all the printing jobs, and the details ofeach printing job as a job information. The details of each printing jobare, for example, the name of a document to be printed, the user name,and the like. The expected execution termination time for all theprinting jobs is calculated in accordance with a number of printingpages for the printing jobs, and the printing processing capacity of theimage forming apparatus 1 for executing the printing jobs, and the like.The job information transmission processing part 16 transmits theacquired job information to the mobile terminal 3 through thecommunication part 12 (Step S21).

Likewise, the job information transmission processing part 16 of thesecond image forming apparatus 1-2 receives a request for transmitting ajob information through the communication part 12. On the basis of theprinting jobs stored in the storage part 13, the job informationtransmission processing part 16 acquires a job information (Step S20).The job information transmission processing part 16 transmits theacquired job information to the mobile terminal 3 (Step S21).

The display part 35 of the mobile terminal 3 receives a job informationfrom the respective first image forming apparatus 1-1 and second imageforming apparatus 1-2 through the communication part 34. On the receivedpieces of job information, the display part 35 performs OSD processingto display them in the display part 35 (Step S22).

At this time, the display part 35 displays the job information about therespective first image forming apparatus 1-1 and second image formingapparatus 1-2, synthesizing it with the image data in association withthe respective disposition location coordinates of the first imageforming apparatus 1-1 and second image forming apparatus 1-2 on thescreen layout. Therefore, in the display part 35, the job information J1about the first image forming apparatus 1-1 and the job information J2about the second image forming apparatus 1-2 are displayed such thatthey are superposed on the first image forming apparatus 1-1 and thesecond image forming apparatus 1-2, respectively, as shown in FIG. 8.Thereby, for example, on the basis of the job information J1, suchcontents as the first image forming apparatus 1-1 having printing jobs;the number of which being five; the expected termination time for thefive printing jobs being 10:35; and in detail of the respective printingjobs, Job 1 being Doc 111, Job 2 being Doc 123, . . . , can be grasped.Likewise, on the basis of the job information J2, that the second imageforming apparatus 1-2 has no printing job, the number of printing jobsbeing zero, can be grasped. By the above operations, the presentprocessing is terminated.

While the first image forming apparatus 1-1 and the second image formingapparatus 1-2 are being image-captured by the photographing part 31, theprocesses from the above-mentioned Step S11 to Step S22 are repetitivelyexecuted. In other words, the job information J1, J2 about therespective first image forming apparatus 1-1 and second image formingapparatus 1-2 that is displayed in the display part 35 is updated inreal time. In addition, the job information J1, J2 may be displayed inthe form of a balloon or a list. Further, the background of the jobinformation J1, J2 may be transparent or translucent.

By the above-described processes, photographing the image formingapparatus 1 with the mobile terminal 3 will provide a superpositiondisplay of the image forming apparatus 1 and the job information aboutthe image forming apparatus 1 in the display part 35. Therefore, theuser can intuitively grasp the relationship between the image formingapparatus 1 and the printing jobs about the image forming apparatus 1through the display part 35. Further, the user can immediately confirmwhat amount of printing jobs is accumulated in the image formingapparatus 1. Further, by photographing a plurality of image formingapparatuses 1 such that they appear in the display part 35, therespective pieces of job information about the plurality of imageforming apparatuses 1 can be grasped simultaneously. Further, since, inthe display part 35, a real image forming apparatus 1 appears, such asituation as that in which some other user is utilizing the imageforming apparatus 1 for copying, or the like, which will not bereflected to the printing job, can be grasped. Therefore, according tothe present embodiment, an image forming apparatus 1 with which printingcan be made the earliest can be selected, whereby the working efficiencycan be improved.

Further, with typical technology, it has been only possible that therelationship between one set of data and another can be displayed in avirtual space, and the relationship between an apparatus in which datais to be saved and the data could not have been intuitively grasped inassociation with the real space. Especially, it has not been possible todisplay a real image forming apparatus together with a job informationabout the printing jobs accumulated in this image forming apparatus.Contrarily to this, according to the present disclosure, it is possibleto provide a technique which allows a real image forming apparatus to bedisplayed together with a job information about the printing jobsaccumulated in this image forming apparatus.

<Embodiment 2>

In the embodiment 1, an example in which, on the basis of a markprovided on the image forming apparatus 1, the image forming apparatusis detected from the image data has been explained. In the embodiment 2,an example in which, on the basis of an appearance figure of the imageforming apparatus 1, the image forming apparatus 1 is detected fromimage data will be explained. Hereinbelow, the points in which theconfiguration of an image forming system 200 in the embodiment 2 isdifferent from that of the image forming system 100 in the embodiment 1will be explained.

Hereinbelow, with reference to FIG. 9, the points in which theconfiguration of a mobile terminal 5 in the embodiment 2 is differentfrom that of the mobile terminal 3 in the embodiment 1 will beexplained.

In the mobile terminal 5, a storage part 53 for storing imagerecognition data 531, an image recognition processing part 52, a displaypart 55, and a request processing part 56 are differ from the storagepart 33 for storing the image recognition data 331, the imagerecognition processing part 32, the display part 35, and the requestprocessing part 36 of the mobile terminal 3 in the embodiment 1, and anazimuth detecting part 54 is newly provided.

The image recognition processing part 52 references the imagerecognition data 531 stored in the storage part 53 for performing imagerecognition processing. In the image recognition processing, the imagerecognition processing part 52 detects an appearance image of the imageforming apparatus 1 from image data captured by the photographing part31. From the detected appearance image, the image recognition processingpart 52 acquires a model information as an apparatus information aboutthe image forming apparatus 1. The image forming apparatus 1 isdifferent in appearance design for each model. Further, the imagerecognition data 531 is constituted such that the pattern of theappearance image of the image forming apparatus 1 is associated with themodel information about the image forming apparatus 1. It is assumedthat the pattern of the appearance image involves not only a pattern ofthe front face of the image forming apparatus 1, but also patterns of aplurality of faces, such as the side face and the rear face.

On the basis of the detection location of the image forming apparatus 1in the image data, the image recognition processing part 52 generates ascreen layout representing the display location of the image formingapparatus 1 on the display screen. Then, the image recognitionprocessing part 52 associates the model information about the imageforming apparatus 1 that has been acquired by the image recognitionprocessing with the disposition location coordinates for the imageforming apparatus 1 in the screen layout.

The azimuth detecting part 54 is comprised of an azimuth sensor, and thelike. The azimuth detecting part 54 measures the terrestrial magnetismfor acquiring a direction of photographing of the photographing part 31.

The request processing part 56 transmits an inquiry request for anaddress information to the server 6 in order to acquire an IP address ofthe image forming apparatus 1 which has been image-captured by thephotographing part 31. The inquiry request for an address information inthe embodiment 2 includes a model information, a screen layout, and adirection of photographing. Among these, the model information is aninformation which has been acquired by the image recognition processingpart 52. The screen layout is an information which has been generated bythe image recognition processing part 52. The direction of photographingis an information about the direction of photographing of thephotographing part 31 that has been acquired by the azimuth detectingpart 54.

By the inquiry request for an address information, the requestprocessing part 56 acquires an IP address of the image forming apparatus1 from the server 6. Then, on the basis of the IP address, the requestprocessing part 56 performs a request for transmitting a job informationto the image forming apparatus 1. As a result of this, the requestprocessing part 56 acquires a job information from the image formingapparatus 1.

The display part 55 displays the job information about the image formingapparatus 1, synthesizing it with the image data in association with thedisposition location coordinates on the screen layout. The display part55 performs synthesizing and displaying about the screen layout in whichthe IP address has been associated when transmitted from the server 6.

Next, with reference to FIG. 10, the points in which the configurationof the server 6 in the embodiment 2 is different from that of the server4 in the embodiment 1 will be explained.

In the server 6, for the address information transmission processingpart 43 and the storage part 42 of the server 4 in the embodiment 1, theaddress information transmission processing part 63 and the storage part62 are different, respectively.

In the storage part 62, unlike the server 4 in the embodiment 1, a roomlayout information 621 is stored, being associated with an addressinformation 422. The room layout information 621 is a room layoutshowing the disposition location and azimuth of the respective first tonth image forming apparatuses 1-1 to 1-n in each actual room.

The address information transmission processing part 63 receives aninquiry request for an address information from the mobile terminal 5through the communication part 41. When the inquiry request for anaddress information is received, the address information transmissionprocessing part 63 references the storage part 62 to identify whichimage forming apparatus 1 in the room layout is the image formingapparatus 1 that has been image-captured by the mobile terminal 5. Theaddress information transmission processing part 63 returns an IPaddress of the identified image forming apparatus 1 to the mobileterminal 5. The specific contents of the processing by the addressinformation transmission processing part 63 will be described later.

The flow of processing in the image forming system 200 as configuredabove will be explained with reference to FIG. 11. Here, forexplanation, it is assumed that the user photographs the first imageforming apparatus 1-1 and the second image forming apparatus 1-2 throughthe mobile terminal 5.

The photographing part 31 of the mobile terminal 5 photographs the firstimage forming apparatus 1-1 and the second image forming apparatus 1-2(Step S41). Thereby, image data covering the first image formingapparatus 1-1 and the second image forming apparatus 1-2 is generated.Then, in the display part 55, an image is displayed on the basis of theimage data captured by the photographing part 31. The user canphotograph the first image forming apparatus 1-1 and the second imageforming apparatus 1-2 while viewing the display part 55.

The image data captured by the photographing part 31 is outputted to theimage recognition processing part 52. The image recognition processingpart 52 references the image recognition data 531 stored in the storagepart 53 for performing image recognition processing. In this imagerecognition processing, the image recognition processing part 52 detectsthe first image forming apparatus 1-1 and the second image formingapparatus 1-2 from the image data which has been acquired from thephotographing part 31 (Step S42).

The image recognition processing part 52 acquires a model informationabout the respective first image forming apparatus 1-1 and second imageforming apparatus 1-2 which have been detected by the image recognitionprocessing (Step S43).

Further, from the image data, the image recognition processing part 52generates a screen layout on the basis of the locations where the firstimage forming apparatus 1-1 and the second image forming apparatus 1-2have been detected, respectively (Step S44). The screen layoutrepresents the locations where the first image forming apparatus 1-1 andthe second image forming apparatus 1-2 are displayed on the displayscreen, respectively. As shown in FIG. 12, the model information aboutthe respective first image forming apparatus 1-1 and second imageforming apparatus 1-2 which have been detected is associated with thedisposition location coordinates on the screen layout for the respectivefirst image forming apparatus 1-1 and second image forming apparatus1-2. In FIG. 12, in the disposition location on the screen layout forthe respective first image forming apparatus 1-1 and second imageforming apparatus 1-2, the respective pieces of model information “AAA”and “BBB” are given for convenience.

Next, the azimuth detecting part 54 acquires a direction ofphotographing of the photographing part 31 (Step S45). For the firstimage forming apparatus 1-1 and the second image forming apparatus 1-2which have been image-captured by the photographing part 31, the requestprocessing part 56 transmits an inquiry request for an addressinformation to the server 5 through the communication part 34 (StepS46). Thereby, the model information about the image forming apparatus1, the screen layout, and the direction of photographing are transmittedto the server 6, being included in the inquiry request for an addressinformation. Among these, the model information is a model informationabout the respective first image forming apparatus 1-1 and second imageforming apparatus 1-2 which have been image-captured by thephotographing part 31. Further, the screen layout shows the dispositionlocation of the respective first image forming apparatus 1-1 and secondimage forming apparatus 1-2. Further, the direction of photographingindicates the direction of photographing of the photographing part 31.

The address information transmission processing part 43 of the server 6receives the inquiry request for an address information from the mobileterminal 5 through the communication part 41. On the basis of thisinquiry request, the address information transmission processing part 63identifies which image forming apparatus 1 which has been image-capturedby the mobile terminal 5 is the image forming apparatus 1 within theroom layout (Step S47). Specifically, when the address informationtransmission processing part 63 receives the inquiry request for anaddress information, it acquires the model information, the screenlayout, and the direction of photographing about the image formingapparatus 1; and they are included in the inquiry request for an addressinformation. Next, the address information transmission processing part63 references the storage part 62 to identify an image forming apparatus1 in the room layout that has a model information which matches to themodel information in the inquiry request for an address information. Ofthe identified image forming apparatuses 1, the address informationtransmission processing part 63 identifies an image forming apparatus 1which is located in an azimuth matching to the direction ofphotographing, and which is located within the range of photographing ofthe photographing part 31, when the image forming apparatus 1 is viewedfrom inside of the room. Further, the address information transmissionprocessing part 63 identifies an image forming apparatus 1 for which thedisposition location of the image forming apparatus 1 in the room layoutcorresponds to the disposition location of the image forming apparatus 1in the screen layout.

For example, it is assumed that, from the mobile terminal 5, as aninquiry request for an address information, there has been transmittedthe direction of photographing of “north”, the screen layout in whichthe first image forming apparatus 1-1 and the second image formingapparatus 1-2 are disposed (see FIG. 12), and the apparatus informationof model AAA and model BBB.

It is further assumed that, in the room layout of the room layoutinformation 621, as shown in, for example, FIG. 13, the first imageforming apparatus 1-1, the second image forming apparatus 1-2, and thenth image forming apparatus 1-n are disposed in the room, beingassociated with the azimuths. In the room layout, the model of the firstimage forming apparatus 1-1 is AAA, the model of the second imageforming apparatus 1-2 is BBB, and the model of the nth image formingapparatus 1-n is NNN.

Here, the image forming apparatuses in the room layout that have a modelinformation which matches to the model information in the inquiryrequest for an address information are the first image forming apparatus1-1 and the second image forming apparatus 1-2. When viewed from insideof the room in the room layout, the first image forming apparatus 1-1and the second image forming apparatus 1-2 are located in the azimuthmatching to the direction of photographing, and are located within therange of photographing. At this time, the disposition locations of thefirst image forming apparatus 1-1 and the second image forming apparatus1-2 in the room layout correspond to the disposition locations of thefirst image forming apparatus 1-1 and the second image forming apparatus1-2 in the screen layout, respectively. In other words, in the screenlayout, on the obliquely forward side of the first image formingapparatus 1-1, the second image forming apparatus 1-2 is disposed,while, also in the room layout, on the obliquely forward side of thefirst image forming apparatus 1-1, the second image forming apparatus1-2 is disposed.

Thus, the address information transmission processing part 63identifies, from among the image forming apparatuses 1 in the roomlayout that have a model information which matches to the modelinformation in the inquiry request for a address information, the firstimage forming apparatus 1-1 and the second image forming apparatus 1-2as the image forming apparatus 1 which, when the image forming apparatus1 is viewed from inside of the room, is located in the azimuth matchingto the direction of photographing, and is located within the range ofphotographing of the photographing part 31, and with which thedisposition location of the image forming apparatus 1 in the room layoutcorresponds to the disposition location of the image forming apparatus 1in the screen layout. Accordingly, even if an inquiry request for anaddress information includes a plurality of image forming apparatuses 1of the same model, the address information transmission processing part63 can identify from among the image forming apparatuses 1. In otherwords, on the basis of the disposition relationship among the imageforming apparatuses 1 in the screen layout, the address informationtransmission processing part 63 can identify an image forming apparatus1 which has been image-captured by the photographing part 31 from amongthe image forming apparatuses 1 in the room layout information 621.

The address information transmission processing part 63 identifies theimage forming apparatus 1 (YES in Step S48). In this case, the addressinformation transmission processing part 63 references the storage part62 to acquire an IP address of the respective identified first imageforming apparatus 1-1 and second image forming apparatus 1-2 (Step S50).When the IP address has been acquired, the address informationtransmission processing part 63 associates the IP address of therespective first image forming apparatus 1-1 and second image formingapparatus 1-2 with the disposition location coordinates. The dispositionlocation coordinates are coordinates of the respective first imageforming apparatus 1-1 and second image forming apparatus 1-2 in thescreen layout which has been acquired from the mobile terminal 5. Then,the address information transmission processing part 63 transmits thedisposition location coordinates and the IP address which has beenassociated therewith to the mobile terminal 5 through the communicationpart 41 (Step S51). Thereby, as shown in FIG. 14, the dispositionlocation coordinates of the respective first image forming apparatus 1-1and second image forming apparatus 1-2 in the screen layout aretransmitted to the mobile terminal 5, being associated with therespective IP addresses. Accordingly, the mobile terminal 5 candistinguish to which image forming apparatus 1 in the screen layout theIP address having been transmitted from the address informationtransmission processing part 63 corresponds. In other words, since thatwhich is performed is more than simply associating the IP address withthe model information, a plurality of image forming apparatuses 1 whichhave been image-captured can be distinguished from one another, even ifthey are of the same model.

The request processing part 56 of the mobile terminal 5 receives the IPaddress of the respective first image forming apparatus 1-1 and secondimage forming apparatus 1-2 from the server 6 through the communicationpart 34. The received IP address is associated with the dispositionlocation coordinates of the respective first image forming apparatus 1-1and second image forming apparatus 1-2 in the screen layout. Further, onthe basis of the received IP address, the request processing part 56performs a request for transmitting a job information to the respectivefirst image forming apparatus 1-1 and second image forming apparatus 1-2through the communication part 34 (Step S52).

The job information transmission processing part 16 of the first imageforming apparatus 1-1 receives the request for transmitting a jobinformation through the communication part 12. Then, the job informationtransmission processing part 16 of the first image forming apparatus 1-1acquires a job information on the basis of the printing jobs stored inthe storage part 13 (Step S53). The job information transmissionprocessing part 16 transmits the acquired job information to the mobileterminal 5 through the communication part 12 (Step S54).

Likewise, the job information transmission processing part 16 of thesecond image forming apparatus 1-2 receives the request for transmittinga job information through the communication part 12. The job informationtransmission processing part 16 of the second image forming apparatus1-2 acquires a job information on the basis of the printing jobs storedin the storage part 13 (Step S53), and transmits the job information tothe mobile terminal 5 (Step S54).

The display part 55 of the mobile terminal 5 receives the jobinformation from the respective first image forming apparatus 1-1 andsecond image forming apparatus 1-2 through the communication part 34.The display part 55 performs OSD processing on the respective pieces ofjob information to display them (Step S55).

At this time, the display part 55 displays the job information about therespective first image forming apparatus 1-1 and second image formingapparatus 1-2, synthesizing it with the image data. In doing this, thedisplay part 55 performs synthesis with the image data in associationwith the disposition location coordinates of the respective first imageforming apparatus 1-1 and second image forming apparatus 1-2 on thescreen layout with which the IP address of the respective first imageforming apparatus 1-1 and second image forming apparatus 1-2 have beenassociated. Thereby, in the display part 55, the job information aboutthe first image forming apparatus 1-1 and the job information about thesecond image forming apparatus 1-2 are displayed such that they aresuperposed on the first image forming apparatus 1-1 and the second imageforming apparatus 1-2, respectively. While the first image formingapparatus 1-1 and the second image forming apparatus 1-2 are beingimage-captured by the photographing part 31, the processes from theabove-mentioned Step S41 to Step S55 are repetitively executed.

There may be a case where, among the image forming apparatuses 1 in theroom layout, there are a plurality of image forming apparatuses 1 whichhave a model information matching to the model information in theinquiry request for an address information, and any one of these islocated in an azimuth matching to the direction of photographing, and islocated within the range of photographing. In this case, there is thepossibility that the address information transmission processing part 63may not be able to identify which image forming apparatus 1 in the roomlayout is the image forming apparatus 1, which has been image-capturedby the mobile terminal 5.

In the case where the image forming apparatus 1 could not have beenidentified (NO in Step S48), the address information transmissionprocessing part 63 notifies the mobile terminal 5 of that the imageforming apparatus 1 could not have been identified, and notifies anerror message to the mobile terminal 5 (Step S49). In doing so, theaddress information transmission processing part 63 may notify themobile terminal 5 of a message for prompting of changing thephotographing object in order to identify the image-captured imageforming apparatus 1. For example, the address information transmissionprocessing part 63 may notify the mobile terminal 5 of a message forprompting of photographing some other image forming apparatus 1 having adifferent model information together with the image forming apparatus 1which could not have been identified. Alternatively, the addressinformation transmission processing part 63 may notify the mobileterminal 5 of a message for prompting of collectively photographing allthe image forming apparatuses having the same model information. Inresponse to the error message, the user may again perform photographing(Step S41).

<Embodiment 3>

In the above-described embodiments 1 and 2, by performing imagerecognition processing on the image data, an apparatus information aboutthe image-captured image forming apparatus 1 has been acquired toinquire of the server about an address information. However, anappropriate method other than this may be used to inquire of the serverabout an address information about the image-captured image formingapparatus 1. For example, by a method which will be explained below, themobile terminal may inquire of the server about an address informationabout the image-captured image forming apparatus 1. Hereinbelow, animage forming system 300 which is constituted by a mobile terminal 6, aplurality of image forming apparatuses 7, and a server 8 will be used asan example for explanation.

The mobile terminal 6, the image forming apparatuses 7, and the server 8are wirelessly connected to one another through an access point. Whilethe image forming apparatuses 7 which have been image-captured by thephotographing part 31 are being appearing on the display part 35, theuser operates an operation key provided on the mobile terminal 6.Operation of this operation key is performed to display the jobinformation about the image-captured image forming apparatuses 7. Uponreceiving the operation of the operation key, the mobile terminal 6transmits an electric field intensity measuring request to the accesspoint. The access point broadcasts the electric field intensitymeasuring request to the image forming apparatuses 7. In response to theelectric field intensity measuring request, the image formingapparatuses 7 broadcast a measurement signal. An apparatus forbroadcasting a measurement signal may be provided in the room. On thebasis of the measurement signal of the image forming apparatuses 7, themobile terminal 6 measures the reception electric field intensity of aradio wave which has been transmitted from each image forming apparatus7. Further, the mobile terminal 6 acquires a direction of photographingthrough the azimuth detecting part 54. For the image-captured imageforming apparatuses 7, the mobile terminal 6 transmits an inquiryrequest for an address information including the reception electricfield intensity measured at the photographing point and the direction ofphotographing by the azimuth detecting part 54 to the server 8 via theaccess point.

In a storage part 82 of the server 8, an address information 422 isstored, being associated with a room layout information 821 whichassociates the actual disposition locations and the azimuths of theimage forming apparatuses 7 in each room with one another. In this roomlayout information 821, the electric field intensity of the radio wavetransmitted by each image forming apparatus 7 installed in the room isfurther indicated. This electric field intensity is that which has beenpreviously measured at each point in the room. The server 8 receives thereception electric field intensity at the photographing point and thedirection of photographing from the mobile terminal 6. Then, the server8 references the storage part 62 to identify a location in the roomlayout corresponding to the reception electric field intensity at thephotographing point. Next, the server 8 identifies a particular imageforming apparatus 7 which, when an azimuth in the room layoutcorresponding to the direction of photographing is viewed from theidentified location in the room layout, is located in a direction ofthat azimuth. Next, for the identified image forming apparatus 7, on thebasis of the disposition location in the room layout, the server 8generates a screen layout at the time when the image forming apparatus 7is displayed in the mobile terminal 6. Then, the server 8 references theaddress information 422 to acquire an IP address of the identified imageforming apparatus 7. The server 8 associates the IP address with thedisposition location coordinates of the image forming apparatus 7 on thegenerated screen layout to transmit it to the mobile terminal 6.

On the basis of the IP address which has been acquired from the server8, the mobile terminal 6 transmits a request for transmitting a jobinformation to the image forming apparatus 7. The mobile terminal 6displays the job information which, on the request for transmitting ajob information, has been acquired from the image forming apparatus 7,synthesizing it with the disposition location coordinates on the screenlayout with which the IP address has been associated. Thereby, byphotographing the image forming apparatus 7, the mobile terminal 6 canprovide a superposition display of the image forming apparatus 7 and thejob information about the image forming apparatus 7.

The present disclosure is not limited to the above-describedembodiments, and of course it is possible to alter them as appropriatewithin the scope of the gist of the present disclosure.

For example, more than one of the above-described embodiments 1 to 3 maybe combined with one another. For example, the embodiment 1 and theembodiment 2 may be combined with each other. In this case, if, on thebasis of the mark provided on the image forming apparatus, theidentification number of the image forming apparatus has been acquired,the processing of the embodiment 1 is executed. Further, if the markprovided on the image forming apparatus has not been detected, and, onthe basis of the appearance image of the image forming apparatus, themodel information about the image forming apparatus has been acquired,the processing of the embodiment 2 may be executed.

What is claimed is:
 1. An image forming system comprising a mobileterminal, a plurality of image forming apparatuses that isimage-captured by a photographing part provided in said mobile terminal,and a server, wherein in a server storage part provided in said server,address information about said plurality of image forming apparatusesbeing stored, when said plurality of image forming apparatuses isimage-captured by the photographing part provided in said mobileterminal, an inquiry request for address information about animage-captured image forming apparatus of said plurality beingtransmitted from said mobile terminal to said server, wherein saidinquiry request for address information includes: a) model informationabout said image forming apparatus, wherein the model information hasbeen acquired by an image recognition processing part provided in saidmobile terminal on the basis of an appearance image of said imageforming apparatus in image data which has been image captured by saidphotographing part, b) a screen layout representing dispositionlocations of respective image forming apparatuses of said plurality ofimage forming apparatuses on a display screen on the basis of detectedlocations of said plurality of image forming apparatus, wherein thedetected location have been detected by said image recognitionprocessing part from said image data, and c) an image-capturingdirection for said photographing part, wherein the direction has beendetected by an azimuth detection part provided in said mobile terminal,said address information stored in the server storage part of saidserver being associated with: a) room layout information thatcorresponds disposition locations of said plurality of image formingapparatuses to azimuths in each actual room, and b) the modelinformation about the image forming apparatuses, which is in the roomlayout information, and being stored, to respond to said inquiry requestfor address information, said server storage part being referenced andthe image-captured image forming apparatus being identified, and, fromsaid server, address information about said identified image formingapparatus being transmitted to said mobile terminal, wherein theimage-captured image forming apparatus: a) is among image formingapparatuses in a room layout that have model information matching tosaid model information in said inquiry request for address information,b) when said image forming apparatuses are viewed from inside of a room,is located in an azimuth matching to said direction of image-capturing,and c) is located within an image-capturing range of the photographingpart, wherein the disposition location of said image forming apparatusin the room layout corresponds to the disposition location of said imageforming apparatus in said screen layout, and address information aboutsaid identified image forming apparatus being transmitted to said mobileterminal, on the basis of said address information, which is transmittedfrom said server and is about said identified image forming apparatus, arequest for transmitting job information being transmitted from saidmobile terminal to said identified image forming apparatus, and torespond to said request for transmitting job information, jobinformation about printing jobs accumulated in said identified imageforming apparatus being transmitted from said identified image formingapparatus to said mobile terminal, and the job information transmittedfrom said identified image forming apparatus being synthesized withimage data image-captured by said photographing part and being displayedin a display part of said mobile terminal, wherein, on the basis ofdisposition relationship among the plurality of image formingapparatuses in the screen layout, said server identifies said imageforming apparatus for which the inquiry request was transmitted byidentifying an image forming apparatus for which its dispositionlocation in the room layout corresponds to the disposition location, inthe screen layout, of the image forming apparatus for which the inquiryrequest was transmitted.
 2. The image forming system according to claim1, wherein said inquiry request for address information includes anidentification number of said image forming apparatus on the basis of amark provided on said image forming apparatus in the image data whichhas been prepared by said photographing part, in the server storage partprovided in said server, the address information about said plurality ofimage forming apparatuses and identification numbers thereof beingassociated with one another and being stored, to respond to said inquiryrequest for address information, said server storage part beingreferenced and being identified an image forming apparatus correspondingto said identification number, and address information about saididentified image forming apparatus being transmitted to said mobileterminal.
 3. The image forming system according to claim 1, wherein saidaddress information stored in the server storage part of said serverbeing associated room layout information, which shows dispositionlocations and azimuths of said plurality of image forming apparatus arecorresponded to an electric field intensity of a radio wave transmittedby each image forming apparatus in each actual room, with an apparatusinformation about the image forming apparatuses in the layoutinformation, and being stored, said inquiry request for addressinformation including a reception electric field intensity for the imageforming apparatus at an image-capturing point, and a direction ofimage-capturing by said photographing part that has been detected by theazimuth detection part provided in said mobile terminal, and to responseto said inquiry request for address information, by said server, beingreferenced said server storage part and being identified an imageforming apparatus which, in a location in said room layout thatcorresponds to said reception electric field intensity at saidimage-capturing point, when an azimuth corresponding to saidimage-capturing direction is viewed, is located in a direction of theazimuth, and address information about said identified image formingapparatus being transmitted to said mobile terminal.
 4. The imageforming system according to claim 1, wherein said job information, whichhas been transmitted from said image forming apparatus, is displayed inassociation with a location where the corresponding image formingapparatus is displayed in said display part of said mobile terminal. 5.The image forming system according to claim 1, wherein said jobinformation includes existence/non-existence of printing job, a numberof printing jobs, an expected execution termination time for all theprinting jobs, and the details of each printing job.
 6. The imageforming system according to claim 1, wherein said server, when saidimage forming apparatus has not been identified, notifies said mobileterminal with an error message.
 7. The image forming system according toclaim 6, wherein said server, when said image forming apparatus has notbeen identified, notifies the mobile terminal with at least one messageselected from: a message for prompting of changing photographing objectin order to identify image-captured image forming apparatus, a messagefor prompting of photographing some other image forming apparatus havinga different model information together with said image forming apparatushaving not been identified, and a message for prompting of collectivelyphotographing all image forming apparatuses having the same modelinformation.